Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅ 📢 Thoughts on this report? Let us know! |
EricGustin
left a comment
EricGustin
left a comment
There was a problem hiding this comment.
This is awesome & great coverage of the API.
One thing to think about: From what the evals are telling me, language models don't seem to fully grasp that they can provide multiple workspace_ids, or multiple tags in a single request. How can the annotations be improved to clear up this confusion?
|
This is an area for improvement. I tried to make some tools more ergonomic. Like the Even 4o is a bit flaky when taking advantage of this ergonomy. Sometimes, it'll invoke So far, I haven't found a way to make them consistent through the argument annotation or the function docstring. Did any ideas pop to your mind that we could try? |
Having them as tools was confusign the LLMs when calling create_task, update_task, or search_tasks. Instead of passing a tag name as arg to create_task, for instance, the LLM was thinking it was supposed to call search_tags_by_name first, which is unnecessary.
|
The |
2 participants
Sorry this first version of the toolkit ended up getting too big. Task is the most relevant object, but it's 'entangled' with everything else in Asana. I realized the toolkit wouldn't be of much value if we didn't have some basic tools related to projects, workspaces, teams, users, and tags as well.